Bellman Error Based Feature Generation Using Random Projections
نویسندگان
چکیده
منابع مشابه
Bellman Error Based Feature Generation using Random Projections on Sparse Spaces
This paper addresses the problem of automatic generation of features for value function approximation in reinforcement learning. Bellman Error Basis Functions (BEBFs) have been shown to improve policy evaluation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algorithm based on random projections, which generates BEBFs for sparse feature spaces....
متن کاملCompressive Reinforcement Learning with Oblique Random Projections
Compressive sensing has been rapidly growing as a non-adaptive dimensionality reduction framework, wherein high-dimensional data is projected onto a randomly generated subspace. In this paper we explore a paradigm called compressive reinforcement learning, where approximately optimal policies are computed in a lowdimensional subspace generated from a high-dimensional feature space through rando...
متن کاملBellman Error Based Feature Generation using Random Projections on Sparse Spaces Appendix
Let x1, . . . ,xn be a time-homogeneous Markov chain with transition kernel T (·|·) taking values in some measurable space X . Consider the concentration of the average of the Markov Process: (x1, f(x1)), . . . , (xn, f(xn)), (1) where f : X → [0, b] is a fixed measurable function. To arrive at a concentration inequality, we need a characterization of how fast (xi) forgets its past. Let T (·|x)...
متن کاملOn the Construction of Initial Basis Function for Efficient Value Function Approximation
We address the issues of improving the feature generation methods for the value-function approximation and the state space approximation. We focus the improvement of feature generation methods on approaches based on the Bellman error. The original Bellman-error-based approaches construct the first basis function as an arbitrary nonzero vector. This kind of design results an inefficient generati...
متن کاملLeast-Squares Regression on Sparse Spaces
Another application is when one uses random projections to project each input vector into a lower dimensional space, and then train a predictor in the new compressed space (compression on the feature space). As is typical of dimensionality reduction techniques, this will reduce the variance of most predictors at the expense of introducing some bias. Random projections on the feature space, alon...
متن کامل